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FINAL  REPORT 


Introduction 

Terminal  guidance  of  weapons  immediately  prior  to  impact  is  a  demanding  control 
problem.  While  Global  Positioning  System  (GPS)  guidance  can  get  a  weapon  close  to  a  target,  a 
greater  level  of  accuracy  is  often  needed  to  account  for  such  uncontrollable  factors  as  the 
movement  of  a  target.  GPS  signals  can  also  be  jammed,  or  the  GPS  receiver  may  be  damaged  as 
it  approaches  a  defended  area.  The  inclusion  of  an  alternate  terminal  guidance  ability  therefore 
will  increase  the  chances  of  the  missile  reaching  its  target. 

A  guidance  system  suitable  for  missiles  must  be  fast,  inexpensive,  and  capable  of 
accurately  recognizing  a  target  under  varying  conditions  including  ambient  weather  and 
changing  aspect  ratio  and  range,  and  in  the  presence  of  clutter  or  countermeasures.  Real-time 
translation  of  the  image  data  into  control  signals  for  the  weapon  is  also  required. 

In  this  Small  Business  Innovation  Research  (SBIR)  Phase  I  project,  QEI  Technologies, 
Inc.  endeavored  to  develop  a  novel,  extremely  high  speed  image-recognition  control  system  that 
will  seamlessly  link  an  image  sensor  input,  target  identification  and  clutter  discrimination 
processes,  and  control  outputs.  The  missile  guidance  architecture  evaluated  in  this  project  uses  a 
novel  mathematical  methodology  whose  hardware  realization  was  designed  to  result  in  an 
extremely  fast,  novel,  and  inexpensive  pattern  recognition  and  control  system. 

The  guidance  system  compares  input  “symbols”  (data  in  symbolic  format  from  an 
imaging  sensor)  to  a  set  of  symbols  stored  in  matrix  form.  As  the  input  image  changes  (due  to 
relative  motion  between  the  image  and  weapon),  a  set  of  continuous  resonant  feedback  patterns 
is  set  up  between  the  input  image  and  the  stored  target  images.  Output  from  this  resonance 
feedback  is  fed  into  a  final  resonant  loop  for  generation  of  the  control  pattern  to  guide  the 
weapon.  The  block  diagram  in  Figure  1  illustrates  the  operation  of  the  control  system. 

As  can  be  seen  in  the  figure,  no  microprocessor  or  other  digital  processing  is  required. 
The  system  instead  operates  by  matching  the  set  of  input  “symbols”  -  i.e.,  data  in  symbolic 
format  from  an  imaging  seeker  -  to  a  stored  set  in  a  second  matrix.  This  concept  can  be 
visualized  as  storing  a  holographic  image  in  digital  matrix  form  and  generating  a  pattern  of  the 
input  image  on  some  projection  of  the  stored  matrix  via  the  generation  of  resonant  patterns.  As 
the  input  image  changes  (due  to  relative  motion  between  the  target  and  weapon),  a  continuous 
resonant  feedback  is  set  up  in  the  first  loop  between  the  input  image  and  stored  “holographic” 
image  matrices.  This  loop  will  result  in  mapping  to  a  set  of  symbols  that  is  fed  into  the  second 
resonant  loop  for  generation  of  a  control  pattern  to  guide  the  weapon. 
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This  section  generates  a  resonant  pattern 
based  on  input  and  stored  image  data 
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The  overall  objective  of  this  three-phase  program  is  to  develop  a  high-speed  image- 
recognition  control  system  for  terminal  guidance  of  a  variety  of  weapons.  In  Phase  I,  the  effort 
focused  on  mathematically  modeling  the  proposed  control  system  as  coupled  to  an  imaging 
sensor.  This  report  describes  the  results  of  the  Phase  I  effort,  including  information  on  the 
following  topics: 


•  Phase  I  Technical  Objectives, 

•  Work  Plan, 

•  Results  by  task 

•  Recommendations  and  Conclusions 


Technical  Objectives 


Phase  I  technical  objectives  are  listed  below: 

•  Mathematically  model  the  imaging  seeker  data  output,  control  matrices,  and  output 
matrices  including  semiconductor  component  delay  times 

•  Test  the  control  system  response  to  a  variety  of  input  conditions,  including  rapidly 
changing  scenarios  and  targets  in  clutter 

•  Generate  a  Bill  of  Materials  for  producing  a  hardware  implementation  of  this  control 
system  using  off-the-shelf  components 

Completion  of  these  technical  objectives  allowed  the  following  questions  to  be  answered, 
thereby  enabling  a  determination  of  feasibility  to  be  made: 

•  Does  the  proposed  control  system  appropriately  recognize  targets,  including  in  cluttered 
conditions? 

•  Does  the  system  generate  appropriate  outputs  to  track  and  pursue  targets? 

•  Are  there  any  conditions  that  can  cause  the  control  system  to  lock  up  or  otherwise  stop 
following  a  target? 
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Phase  I  Work  Plan 


The  Phase  I  work  plan  is  divided  into  four  technical  tasks.  These  tasks,  together  with 
their  associated  objectives,  are  listed  below. 


Task  1.  Mathematically  Model  Symbolic  Control  System  Coupled  to  an  Imaging  Seeker 

The  objective  of  this  task  is  to  mathematically  model  the  symbolic  control  system 
including  simulated  image  inputs. 

A  model  for  the  entire  control  system  was  developed  using  MathCAD  software  using 
symbolic  notation.  The  model  is  based  on  simpler  but  proven  models  of  this  control  system  that 
the  PI  has  used  in  the  past  (Sullivan  1999).  The  model’s  configuration  is  illustrated  in  Figure  2 
and  described  below. 

The  model  consists  of  the  following  elements: 

Light  Detection  Array.  The  array  is  fed  predetermined  test  patterns,  and  the  array 
generates  a  simulated  pixel-by-pixel  response  to  the  test  pattern.  Each  pixel  of  the  array  (256  x 
256  pixels  for  simulations)  is  directly  coupled  to  an  analog  to  discrete  symbolic  converter.  Each 
converter  possesses  256  discrete  output  lines  (corresponding  to  an  8-bit  level  of  image 
resolution),  and  only  one  output  line  is  selected  based  on  the  value  of  the  analog  input.  This 
serves  to  simulate  the  fastest  responses  the  control  system  may  achieve,  and  serves  as  an  accurate 
representation  of  the  data  flow  to  be  used  in  Phase  III  systems. 

Image  Recognition  Matrices.  A  set  of  mapped  matrices  with  built-in  time  delays  are 
created  to  perform  the  pattern  matching  and  to  store  specific  patterns  to  be  recognized. 

The  patterns  are  stored  in  a  manner  analogous  to  a  holographic  approach,  so  it  is  not  necessary  to 
store  an  exact  pattern  for  every  aspect  ratio  of  a  target. 

Control  Matrices.  A  second  set  of  mapped  matrices  with  built-in  time  delays  convert 
the  recognized  pattern  (as  output  by  the  image  recognition  matrices)  into  a  standard  control 
pattern  whose  shape  and  position  provides  control  to  the  missile. 

Output  Conversion.  The  standard  control  pattern  is  converted  into  a  single  aiming  point 
on  a  matrix-based  output  grid. 
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Analog  or  Digital  to 
Discrete  Symbolic 
Matrix  Converters 


This  section  generates  a  resonant  pattern 
based  on  input  and  stored  image  data 
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Task  2.  Conduct  Simulations  of  the  Symbolic  Control  System. 

The  objective  of  this  task  is  to  conduct  mathematical  simulations  of  the  symbolic  control 
system  performance  using  the  model  developed  in  Task  1. 

The  model  developed  in  Task  1  is  used  to  simulate  the  behavior  of  the  control  system  as 
it  is  presented  with  a  variety  of  input  scenarios.  Various  shapes  were  superimposed  on 
backgrounds  containing  various  sorts  of  clutter.  Specific  tests  include  the  following: 

Basic  Pattern  Recognition.  The  system  must  first  be  evaluated  in  terms  of  how  well  and 
how  quickly  it  recognizes  basic  shapes  in  a  clutter-free  environment.  The  shape  size  and 
orientation  are  varied  during  these  tests. 

Multiple  Patterns.  The  system  is  presented  with  multiple  patterns  in  the  same  image 
and  evaluated  in  terms  of  its  ability  to  select  the  appropriate  one  to  pursue. 

Noise  and  Clutter.  The  effect  of  noise  and  clutter  on  individual  pattern  recognition  will 
be  evaluated  using  both  random  and  patterned  backgrounds. 

Figure  3  is  a  pictorial  representation  of  how  simulation  data  enter  the  model  system, 
generate  resonant  patterns,  and  output  control  signals. 
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4.  Generation  of  a  single 
aim  point  to  guide  the 
missile  (missile  is  assumed 
to  be  pointed  at  the  center 
of  the  matrix) 


3.  Generation  of  standard 
control  pattern  from  above 
matrix  and  stored  control 
patterns 


Figure  3.  Pictorial  Representation  of  Data  Processing  in  the  Symbolic  Control  System. 


Task  3.  Hardware  Analysis. 

The  objective  of  this  task  is  to  begin  selecting  appropriate  components  that  can  be  used 
to  implement  the  control  system. 

This  begins  the  process  of  identifying  suitable  parts  with  which  to  build  the  control 
system.  Results  from  this  search  will  allow  predictions  to  be  made  of  near-term  and  long-term 
anticipated  control  system  performance. 
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Task  4.  Perform  Technical  and  Economic  Evaluation. 


The  objective  of  this  task  is  to  complete  an  initial  evaluation  of  the  Phase  I  results  to 
assess  feasibility  and  potential  for  follow-on  development. 

This  task  includes  a  preliminary  technical  and  economic  evaluation  of  the  Phase  I  results. 
This  evaluation  includes  comparing  the  modeled  performance  and  characteristics  of  the  control 
system  against  military  requirements  and  available  electronic  hardware.  Cost  estimates  are  also 
developed  for  production  control  systems. 


Results 

Task  1.  Mathematically  Model  Symbolic  Control  System  Coupled  to  an  Imaging  Seeker 

A  model  for  the  entire  control  system  has  been  developed  using  MathCAD  software. 

This  system  consists  of  the  following  major  elements: 

•  A  program  for  generating  embedded  pattern  targeting  information 

•  A  program  that  generates  the  test  images  (input  program) 

•  The  program  that  performs  the  actual  pattern  recognition  task 

The  input  program  accepts  standard  bitmap  images  for  use  by  the  pattern  recognition 
program.  The  pattern  recognition  program  compares  the  input  matrix  to  the  embedded  image 
matrix  and  generates  an  output  matrix  that  includes  a  correlation  quality  figure  indicating  the 
quality  of  fit  between  the  input  image  and  each  of  the  stored  target  images.  The  image 
recognition  program’s  output  matrix  generates  simulated  control  signals  to  guide  the  missile  to 
the  desired  aim  point  on  the  image. 

Because  of  software  limitations,  the  Phase  I  targeting  system  is  divided  into  two  subsets, 
each  using  a  different  type  of  embedded  pattern  and  mapping  to  different  paths  upon  past/present 
image  comparison.  The  first  subset  is  for  moving  targets  and  assumes  the  background  changes 
measurably  less  than  the  target  and  any  decoy.  The  second  subset  is  for  fixed  targets,  where  the 
target  image,  decoys,  and  background  are  considered  a  semi-fixed  entity. 

The  two  subsets  use  mapping  to  past/present  comparisons  and  null  responses  that  are 
mirrored  to  each  other.  Incorporating  both  into  a  single  system  results  in  the  degradation  of  both. 
In  a  physical  system,  both  subsets  would  be  operating  in  parallel  and  generate  independent 
targeting  information.  The  one  subset  whose  correlation  to  the  embedded  target  is  highest 
provides  the  primary  targeting  point.  This  approach  allows  operation  against  a  target  that  can 
both  move  and  then  remain  motionless.  The  first  half  of  each  subset  is  identical  and  can  use  the 
same  video  processing  substrate. 
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Task  2.  Conduct  Simulations  of  the  Symbolic  Control  System 

The  model  developed  in  Task  1  has  been  used  to  simulate  the  behavior  of  the  control 
system  as  it  is  presented  with  a  variety  of  input  scenarios.  Several  simulations  were  run, 
including  the  one  described  below  and  in  Figure  4. 

The  target  in  this  simulation  is  a  grainy  image  of  a  Humvee  that  is  moving  relative  to  the 
field  of  view  (a  stationary  object  such  as  a  building  would  also  appear  to  be  moving  from  the 
viewpoint  of  a  missile).  The  background  in  this  simulation  is  slightly  cluttered  and  variable. 

The  purpose  of  this  simulation  was  to  evaluate  the  system’s  ability  to  distinguish  a  grainy  and 
somewhat  complex  image  from  simple  but  similarly  shaped  objects  such  as  rectangles  and 
circles.  In  addition,  the  system’s  ability  to  track  the  image  against  a  varying  background  was 
evaluated  by  adjusting  the  background  from  frame  to  frame. 

Figure  4  illustrates  how  the  system  tracks  the  target  image  by  showing  three  things:  1)  the 
input  image,  2)  the  output  of  the  image  recognition  program,  and  3)  the  aim  point.  The  output  of 
the  image  recognition  program  shows  “hits”  in  the  form  of  dots  -  what  is  not  clearly  illustrated 
in  this  figure  is  that  each  dot  represents  a  correlation  quality  figure  indicating  the  quality  of  fit 
between  the  input  image  and  each  of  the  stored  target  images.  This  quality  of  fit  plays  a 
significant  role  in  determining  the  aim  point,  although  in  Phase  I  the  algorithm  for  determining 
the  aim  point  is  fairly  crude.  Nonetheless,  Figure  4  clearly  illustrates  an  accurate  aim  point  for 
this  simulation.  With  the  exception  of  the  last  frame,  the  program  successfully  tracked  the  target. 

A  second  simulation  illustrates  the  case  of  a  stationary  target  (Humvee)  with  moving 
background  images.  One  of  the  moving  images,  a  triangle  consisting  of  lots  of  small  squares  of 
various  shades,  was  used  to  evaluate  the  model’s  ability  to  distinguish  between  the  target  image 
and  a  dissimilar  image  containing  a  large  number  of  edges. 

Figure  5  shows  the  results  of  this  simulation.  In  this  case,  the  moving  triangle  received  a 
more  hits  than  the  target  Humvee,  indicating  that  the  system  was  distracted  by  the  large  number 
of  edges  in  the  moving  triangle.  Modifying  the  system  to  track  a  stationary  target  moves  the  aim 
point  to  the  Humvee.  However,  if  the  Humvee  and  triangle  had  the  same  motion  relative  to  one 
another,  the  system  would  have  trouble  distinguishing  between  them. 
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Figure  4,  Simulation  of  Symbolic  Control  System  Using  Humvee  Target 
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Figure  5.  Simulation  with  stationary  target  and  moving  background  clutter 
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Task  3.  Hardware  Analysis 

Figure  6  illustrates  the  circuit  layout  of  a  basic  pattern  recognition  mapping  module  using 
a  universal  “on  the  fly”  configuration  (middle  set  of  “AND”  gates).  Interfacing  mapping 
modules  to  a  CCD  array  or  to  each  other  requires  a  method  to  handle  the  large  numbers  of 
input/output  mapping  points.  For  optimum  performance,  subcomponents  (CCD,  mapping 
modules,  and  mapping  modules  to  digital  output)  would  need  to  be  incorporated  on  a  single  chip 
using  very  large  scale  (VLS)  integrated  circuit  technology.  This  option  provides  a  lightweight, 
high  performance  system,  but  millions  of  dollars  would  be  required  for  chip  design  and 
fabrication.  This  option  is  not  viable  under  the  SBIR  program. 

A  more  cost  effective  option  would  incorporate  a  set  of  mapping  modules  on  a  single 
chip  with  integral  parallel  type  input/output  ports.  This  approach  allows  the  pattern  recognition 
system  to  be  directly  incorporated  into  existing  systems  at  reduced  cost  and  delay.  QEI’s  initial 
estimate  for  prototype  production  of  this  system  is  $250,000.  Since  several  iterations  of  silicon 
design,  fabrication,  and  testing  are  anticipated  for  successful  system  development,  this  option  is 
also  not  viable  for  a  Phase  II  project. 

A  viable  option  for  a  Phase  II  prototype  system  would  use  embedded  algorithms  on  a 
microprocessor.  This  is  a  cost  effective  approach  that  emulates  the  functionality  of  the  pattern 
recognition,  although  it  sacrifices  the  speed  and  compactness  of  operation  that  would  be 
achieved  by  a  true  integrated  circuit  implementation.  The  Phase  II  system  would  not  be  suitable 
for  field  use  because  of  size  and  memory  constraints. 
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Figure  6.  Circuit  layout  of  a  basic  pattern  recognition  mapping  module  using 
universal  “on  the  fly”  configuration 
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Task  4.  Technical  and  Economic  Evaluation 


If  successful,  this  technology  could  have  widespread  military  and  commercial 
applications.  In  addition  to  employing  the  technology  on  cruise  missiles,  it  could  also  be  used  on 
missile  interceptors,  air-to-air  missiles,  and  camion  projectiles.  Commercially,  the  technology  is 
applicable  to  a  wide  variety  of  visual  control  systems  including  manufacturing  process  control, 
aids  for  the  blind,  vehicle  collision  avoidance  sensors,  autonomous  vehicle  control,  and 
surveillance  systems.  The  technology  would  be  particularly  useful  in  the  control  of  non-linear, 
time-varying  processes  such  as  non-linear  chemical  processes  and  unstable  control  systems.  The 
proposed  control  system  would  be  applicable  to  anything  that  requires  complex  pattern 
recognition  in  a  short  time  frame  or  to  systems  handling  long-term  drift  that  require  quick 
response  times. 

The  simulation  results  indicate  that  the  system  adequately  tracks  a  moving  target  against 
a  stationary  background  with  moving  decoys,  or  a  stationary  target  and  decoys  against  a 
background  that,  as  a  whole,  varies  over  time.  Non-target  decoy  images  must  contain  less  than 
50%  of  the  target  information;  otherwise,  these  images  may  be  identified  as  targets.  This 
limitation  means  the  system  would  probably  not  be  able  to  distinguish  between  similar  target  and 
non-target  objects,  such  as  buildings  or  vehicles.  Significant  improvements  to  the  image 
recognition  capabilities  of  this  system  are  needed  before  it  would  be  useful  to  either  the  military 
or  commercial  markets. 

Cost  is  another  issue.  Initially,  QEI  had  hoped  that  the  proposed  control  system  would  be 
easily  implemented  through  combining  a  digital  image  sensor,  Erasable  Programmable  Read- 
Only  Memory  (EPROM)  chips,  First-In  First-Out  (FIFO)  buffers,  and  Field  Programmable  Gate 
Array  (FPGA)  chips.  Unfortunately,  implementation  is  likely  to  prove  far  more  complex.  As 
described  in  the  previous  section,  optimum  performance  would  require  combining  all 
subcomponents  on  a  single  chip  using  VLS  integrated  circuit  technology.  Even  with  significant 
contributions  from  a  private-sector  partner,  this  option  may  not  be  economically  viable. 
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Conclusions 


All  of  the  technical  objectives  for  this  project  have  been  successfully  completed,  as 
follows: 

•  The  imaging  seeker  data  output,  control  matrices,  and  output  matrices  including 
semiconductor  component  delay  times  were  mathematically  modeled 

•  The  control  system  response  to  a  variety  of  input  conditions,  including  rapidly  changing 
scenarios  and  targets  in  clutter,  was  tested 

•  A  preliminary  evaluation  of  materials  needed  for  producing  a  hardware  implementation 
of  this  control  system  was  completed 

Several  simulations  of  the  control  system  were  completed,  demonstrating  that  the  system  can 
quickly  recognize  targets  under  certain  conditions.  Currently,  non-target  images  must  contain 
less  than  50%  of  the  target  information;  otherwise,  these  images  may  be  identified  as  targets. 

This  is  a  significant  limitation  that  must  be  addressed  before  the  control  system  could  be 
effectively  used  for  military  or  commercial  applications. 

The  most  likely  area  for  improvement  is  the  way  target  images  are  embedded  in  the  system 
and  compared  to  input  images.  Several  options  are  being  considered,  although  at  this  time  none 
of  the  options  have  been  developed  to  the  point  that  they  are  ready  to  be  tested. 

If  these  technical  issues  can  be  adequately  addressed,  the  control  system  would  have 
widespread  military  and  commercial  applications.  For  optimum  performance,  all  subcomponents 
could  be  incorporated  on  a  single  chip  using  very  large  scale  (VLS)  integrated  circuit 
technology.  This  option,  while  expensive,  yields  an  extremely  fast,  lightweight,  and 
performance  control  system. 
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